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ABSTRACT 


As a participant of the year 2000 NASA Summer Faculty Fellowship Program, I 
worked with the engineers of the Dexterous Robotics Laboratory at NASA Johnson 
Space Center on the Robonaut project. The Robonaut is an articulated torso with two 
dexterous arms, left and right five-fingered hands, and a head with cameras mounted on 
an articulated neck. This advanced space robot, now driven only teleoperatively using 
VR gloves, sensors and helmets, is to be upgraded to a thinking system that can find, in- 
teract with and assist humans autonomously, allowing the Crew to work with Robonaut 
as a (junior) member of their team. Thus, the work performed this summer was toward 
the goal of enabling Robonaut to operate autonomously as an intelligent assistant to as- 
tronauts. 

Our underlying hypothesis is that a robot can develop intelligence if it learns a set 
of basic behaviors ( i.e reflexes - actions tightly coupled to sensing) and through experi- 
ence learns how to sequence these to solve problems or to accomplish higher-level tasks. 
We describe our approach to the automatic acquisition of basic behaviors as learning 
sensory-motor coordination (SMC). Although research in the ontogenesis of animals 
(development from the time of conception) supports the approach of learning SMC as the 
foundation for intelligent, autonomous behavior, we do not know whether it will prove 
viable for the development of autonomy in robots. The first step in testing the hypothesis 
is to determine if SMC can be learned by the robot. To do this, we have taken advantage 
of Robonaut’ s teleoperated control system. When a person teleoperates Robonaut, the 
person's own SMC causes the robot to act purposefully. If the sensory signals that the ro- 
bot detects during teleoperation are recorded over several repetitions of the same task, it 
should be possible through signal analysis to identify the sensory-motor couplings that 
accompany purposeful motion. 

In this report, reasons for suspecting SMC as the basis for intelligent behavior will 
be reviewed. A robot control system for autonomous behavior that uses learned SMC 
will be proposed. Techniques for the extraction of salient parameters from sensory and 
motor data will be discussed. Experiments with Robonaut will be discussed and pre- 
liminary data presented. 
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INTRODUCTION 


To interact naturally with people in a human-centered environment, a robot must 
be able to coordinate sensing with action. That is, it must have Sensory-Motor Coordi- 
nation (SMC). It is possible to program a certain degree of SMC into a robot prior to its 
deployment. But it is impossible for a programmer to anticipate every physical contin- 
gency that may arise in a robot's interactions with people. This is due to the intrinsic 
complexity of a human-centered environment. Only animals (including people) have 
SMC that permits them to work effectively in a complex natural world. If SMC in ani- 
mals were well understood - if the structures and functions of the systems that manifest it 
were known - then analogous systems could be implemented in robots. 1 SMC in animals 
is not completely understood, but research in it has recently advanced to the point where 
plausible mechanisms for it have been described. Evidence from studies in neurophysi- 
ology [1], ontogenesis [2,3], and cognitive science [4] suggests that to interact effec- 
tively and efficiently with its environment, an animal must learn through its own 
experiences the reciprocal causative relationships between sensing and action that foster 
its success or survival (cf. below). That is, SMC must be learned, or at least refined, 
through an animal's direct experience with acting in the world. 

Schema theory [4] can be used to describe the functional aspects of an animal's 
behavior without exact specification of the biological systems that support it. Schemas 
exist at a frame of reference higher than that of the individual computational elements 
(neurons in the case of animals). A schema description of the behavior of an animal is 
inherently modular. It provides a framework for the description of behaviors in terms of 
the interactions of modules that control motion, process sensory information, create and 
recall memories, etc. In animals, the modules may more or less directly correspond to 
specific networks of neurons. But this separation of function from structure affords the 
possibility of realizing the behavior of an animal in a robot by substituting computers and 
electro-mechanical devices for neuron networks and bio-mechanical subsystems. Be- 
havior-based robots (BBR) [5,6] are particularly amenable to this. BBRs act through the 
combination of basic behaviors , which are motor actions tightly coupled to sensory stim- 
uli - both external to the robot and internal (i.e., proprioceptic). 

This report proposes a method for the learning of sensory-motor coordination 
through the teleoperation of a behavior-based robot. The goal of the work is to enable a 
robot to leam SMC by finding the correlations between sensory events and motor control 
events that co-occur during task execution. The robot is guided by a human operator 
* through repeated trials of a specific task while recording all its incoming sensory data. 

The motor and sensory data gathered throughout the trials will be analyzed to find repre- 
sentative couplings between sensory stimuli and motor actions. If successful this will not 


1 Implementation is possible if the functionality of the biological systems can be reproduced in 
electro-mechanical systems. Schema theory suggests that it can. (cf. below). 
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only permit the robot to perform the task autonomously, but also (with an appropriate 
control system) enable the robot to adapt to variations in the task or in the environment. 

SENSORY-MOTOR COORDINATION 

Sensory-Motor Coordination underlies the physical behavior of an animal in re- 
sponse to its environment. More than a response, SMC is a feedback loop that changes 
both the animal and the environment. An animal's motions are caused by muscle con- 
tractions. These contractions are elicited by electrochemical signals that are generated by 
circuits of motor neurons. When the animal moves, it causes a relative shift in the envi- 
ronment. As the environment shifts, energy patterns sweep across the animal's sensory 
organs. Sensory organs are transducers that, in effect, transform external, spatio- 
temporally dynamic energy fields into electrochemical signals carried by circuits of sen- 
sory neurons internal to the animal. These sensory signals (more or less directly) modu- 
late the signals in the original motor circuits. Learning occurs in the mapping from 
sensory response signal to motor control signal. Thus, an animal senses the environment 
and acts. The action changes the environment relative to the animal, which senses those 
changes and acts accordingly. 

SMC is likewise needed by a sensory-guided robot. The basic behaviors of a 
BBR are independent units of SMC. They include what are commonly called reflex ac- 
tions. When a basic behavior is enabled 2 and the stimuli associated with it occur, the ac- 
tion is performed - without resort to modeling or deliberation. Basic behaviors are 
canonical in the sense that all actions exhibited by the robot are generated through the 
cooperation and competition of basic behaviors operating concurrently or in sequence. 

At any given point in time, some of the basic behaviors will be enabled and others sup- 
pressed depending on the task and environmental context of the robot. Since a BBR ex- 
hibits any and all its behaviors through the combination and sequencing of basic 
behaviors, a BBR is wholly dependent on, and to a large extent defined by, sensory motor 
coordination. 

Sensory-motor coordination is fundamental for another compelling reason. It 
forms a foundation for higher level learning and perception. In particular, the categori- 
zation of sensory stimuli can be accomplished through SMC [7], A mobile agent can 
learn the sensory patterns that correspond to an obstacle by associating stimuli with its 
motor responses, as when a characteristic stimulus pattern routinely accompanies the 
sudden inability to move. Similarly, as Pfeifer has demonstrated, an agent can leam to 
distinguish between objects that it can manipulate and those which it cannot [8], If the 
internal sensation of a need (a drive or a goal) having been satisfied accompanies a set of 
actions performed in the presence of specific stimuli, that stimuli can be recognized as 


2 A BBR typically has a suite of basic behaviors, not all of which are operational at the same time. 
Depending on the task and environmental contexts, various basic behaviors will be enabled or disabled. If a 
behavior is enabled - made operational - it will remain quiescent until its triggering sensory stimuli are 
present. 
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being beneficial to the agent ( e.g ., an energy source - food). Recent experiments by 
Pfeifer and others have demonstrated that such SMC events can be used to learn classifi- 
cations of objects and events in the environment more easily and more accurately than 
can traditional machine sensing strategies such as model-based vision [9,10]. 

SCHEMA THEORY 

Since the behavior of animals is mediated by their nervous systems, the under- 
standing of their behavior from first principles requires an understanding of nervous sys- 
tems. Neuroscience has provided a structural description that includes neurons 
(individuals and networks) and layers, columns, and modules in the brain [11]. But the 
function of these structures is not completely understood and it is function more than 
structure that determines behavior. Functional analysis is complicated by the fact that 
many of the neuronal structures participate in different functions. With certain excep- 
tions there are no discernible one-to-one mappings of low-level structure to high-level 
function [4], 

Arbib et al. employ schema theory “as a framework for the rigorous analysis of 
[animal] behavior that requires no prior commitment to hypotheses on the location of 
each schema (unit of functional analysis) but can be linked to a structural analysis as and 
when it becomes appropriate.” [4] (p. 33). Thus schemas are descriptions of functions 
that are performed by networks of neurons and the muscles and appendages that they 
control. Schema theory enables the top-down analysis of a complex behavior by provid- 
ing a structure for logically dissembling it, that is it facilitates the analytical decomposi- 
tion of a complex behavior into sets of simpler behaviors. On the other hand, schemas 
also enable the bottom-up analysis of sets co-occurring behaviors. The collective be- 
havior of a set of simple schemas can be deduced if the framework for their competition, 
cooperation, and sequencing is known. This collective behavior is a higher-level schema 
called an assemblage. Not only are the behaviors of animals describable by schemas but 
also are the control systems of behavior-based robots. BBRs are, given their modular ar- 
chitectures, particularly amenable to such description. The theory of behavior-based ro- 
botics is grounded on the idea that complex behavior in an agent emerges through the 
competition and cooperation of simple behaviors in the context of an environment, which 
is precisely the idea of assemblage in schema theory. 

To the extent that function can be separated from structure, a schema representa- 
tion enables a specific behavior to be performed by agents with dissimilar computational 
hardware. In particular, a behavior observed in an animal that can be described accu- 
rately by schemas could be implemented on an appropriately structured robot. Schemas, 
therefore, provide for comparative analysis of similar behaviors on dissimilar agents, be 
they bio-chemical or electro-mechanical. 

Arbib et al. group schemas in two categories. Motor schemas are “the control 
systems which can be coordinated to effect the wide variety of movement. A set of basic 
motor schemas is hypothesized to provide simple, prototypical patterns of movement.” 
Perceptual schemas “are those used for perceptual analysis. They embody the processes 



whereby the system determines whether a given domain of interaction is present in the 
environment. They not only serve as pattern-recognition routines but can also provide 
the appropriate parameters concerning the current relationship of the organism with its 
environment.” [4] (p. 42). 

Research in the ontogenesis of animals has demonstrated that the ability to move 
exists prior to an animal's ability to sense its environment. Arbib et al. state that this 
“does not, however, imply that motility is an end in itself. Rather this, ‘motor founda- 
tion’ serves to group the later development of sensory maps and sensorimotor representa- 
tions in a self-directed manner.” [4] (p. 10). Thus, in animals the formation of the 
musculo-skeletal system and the neuro-circuits for motor control precedes the develop- 
ment of perceptual schemas. Such a development schedule makes sense. Perceptual 
schemas in animals, even if passed on phylogentically, must be tuned; sensory stimuli is 
required for a perceptual modality to develop. Other perceptual schemas ( e.g., a seman- 
tic description of a visual object) must be learned. On the other hand, an animal must, to 
a certain extent, “hit the ground running” to survive. Motion must precede perception so 
that the animal can move at birth and so that the effects of its motion can be perceived 
and learned. Perceptual schemas, must therefore be learned or tuned in concert with mo- 
tion. Simultaneously, motor schemas must be tuned to enable efficient sensing. Thus, 
sensory-motor coordination requires the coupling of perceptual schemas and motor 
schemas into assemblages. Perceptual schemas provide goal and trajectory information 
to the motor schemas, whereas the latter provide a physical framework within which a 
perceptual schema can extract salient information. Arbib et al. place motor schemas and 
perceptual schemas at the foundation of animal function. Under the influence of the en- 
vironment these schemas self-organize to control an animal's behavior. 

For the designers of robots the main implication of the onset of motility prior to 
sensation in animals is that reflexes are not primary. (See [4] Sec. 2.1.1, p. 13 f.f.) Put in 
another way, basic behaviors are not truly basic. Motion is primary; it can happen with- 
out sensing. Reflexes develop with the onset of sensing. Then sensory signals modulate 
the signaling of motor circuits and reflexes emerge. 

SCHEMAS AND SMC IN BEHAVIOR BASED ROBOTS 

The following four examples of behavior-based robot control systems depend on 
SMC and can be described through assemblages of schemas. Each of the architectures 
has basic behaviors at its foundation. In each case, the basic behaviors are selected by the 
designer of the robot. Each of the architectures can be designed to learn, and as a result 
exhibit emergent SMC. The learning, however, occurs at levels above basic behaviors. 

Brooks' subsumption architecture 

Brooks' subsumption architecture controls a robot through a collection of aug- 
mented finite state machines (AFSM) organized into layers [5], A subsumptive robot has 
no central planner or controller. Each AFSM can be activated by sensory inputs and pro- 
duces outputs that drive actuators or are passed to the inputs of other modules. Within 
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subsumption, the AFSMs are motor schemas. The sensory inputs are perceptual sche- 
mas. An AFSM with well-defined sensory input implements a basic behavior. Assem- 
blages are formed dynamically as AFSMs at one level are activated or inhibited by 
AFSMs at a higher level. Usually the basic behaviors in the lowest layer are prepro- 
grammed; the sensory signals that trigger an AFSM are not learned. Learning can take 
place in a subsumption architecture, (e.g., Brooks' robot, Ghengis [12]) but generally this 
occurs in layers above the first. 

Mataric’s action-oriented representations. 

Mataric designed, using subsumption, a mobile robot that learns to navigate an 
environment through the use of action-oriented representations [13]. The robot both gen- 
erates and operates from an “action map” of the environment. While wandering in the 
environment and reacting to sensory input according to its basic behaviors (e.g., wall 
following, object avoidance, etc.) the robot generates the map by building up a directed 
graph. Each node of the graph contains a description of the motor state at the time of its 
formation and description of the sensory data that was received as the robot performed 
the actions described by the motor state. Adjacent nodes in the graph correspond to adja- 
cent areas in the environment. Once the environment has been mapped, the robot can 
reach a physical location by activating the corresponding node of the graph. The graph is 
searched (using spreading activation) back from the goal node to the node that represents 
the current position of the robot. The nodes along the shortest connecting path are en- 
abled. The robot reaches the goal by moving according to the motor commands of its 
current node until its sensory input more closely matches the data from the next node. 
Then it executes the motor commands from next node and proceeds successively from 
node to node until the goal is reached. 

Mataric's robot learns while acting by forming a spatio-temporal sensory-motor 
description of the environment. The map indicates the sensory and motor status of the 
robot at a particular point in space at a particular time relative to the current position. 
Thus, the robot learns how to sequence and basic behaviors from sensory input. This is 
undoubtedly a form of SMC but it learns the sequencing of basic behaviors rather than 
the SMC that defines the basic behaviors themselves. 

Arkin's Motor Schema. 

A robot controlled by Arkin's motor schema 3 architecture follows gradients in a 
vector field map of its environment [14]. Computational modules such collision detec- 
tors and obstacle or object recognizers are perceptual schemas since they compute the 
vectors at points in space that serve to impel the robot. Motor schemas (in Arbib's sense) 
within Arkin's architecture are assemblages of motor controllers that respond individually 
to components of the vector field map. A motor controller generally has a fixed response 


3 Motor Schema is the name that Arkin has given his control architecture. It makes use of both 
perceptual schemas and motor schemas in the sense that Arbib describes them. 
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to its input vector. The response is a function of the magnitude and direction of the input 
vector, but that function is generally preprogrammed and does not change. Any learning 
that occurs happens in the perceptual schemas that compute the vector field. 

Pfeifer's SMC-based categorization. 

Pfeifer's robots are based on an extended Braitenberg architecture, another type of 
BBR [15]. A number of basic behavior modules (Pfeifer calls these “reflexes”) operate 
in parallel, receiving sensory inputs (including proprioception) and summing their out- 
puts onto the motor controllers [8], The response of each behavior module to its inputs is 
preprogrammed. The overall robot system does learn, however, as it interacts with the 
environment, guided by a “value system.” Values are, essentially, the preprogrammed 
reflexes and reinforcement schemes, that cause the robot to seek some sensory stimuli 
and to avoid others. Learning occurs through the adaptive modulation of sensory signals 
that are fed to the behavior modules. 

Pfeifer defines categorization of an object as the robot's appropriate interaction 
with the object. Through the value-based learning scheme the robot learns how to couple 
sensing with actuation so that appropriate behaviors are learned for different stimulus 
patterns. Thus the objects that project the different stimulus patterns are classified de 
facto without forming an abstract model of the object. Pfeifer's robot leams about objects 
by finding the correlations between sensory signals and behaviors that lead to favorable 
results and by decoupling behaviors from stimuli when that coupling leads to unfavorable 
results. Thus, an appropriate linkage between sensing and action at the task level is 
learned by trial and error. 


LEARNING BASIC BEHAVIORS 

Behavior-based robots employ schemas implicitly. Their complex behaviors 
emerge through the interaction of a canonical set of basic behaviors, each of which is a 
sensory-driven motor controller. Therefore, in a BBR high-level behavior emerges from 
assemblages of perceptual schemas linked to motor schemas, just as in animals. In terms 
of schema theory, the practice of designing BBRs differs from the ontogenesis of ani- 
mals. The designer of a BBR must decide ad hoc or through trial and error, exactly 
which coupling of sensory data to motor controller constitutes a useful basic behavior. 
And the designer must decide which basic behaviors to include in the canonical set. He 
or she determines the perceptual to motor schema linkage at the base level and decides 
which of these first-order assemblages to include on the robot. 4 In other words, the de- 
signer programs SMC into the robot at the lowest level. 

BBRs that leam, such as those described in the previous section, learn at the level 
above basic behaviors. They leam which behaviors to activate and which to inhibit or to 
suppress under various sensory conditions, or they leam an appropriate sequence of be- 


4 In some BBRs the higher-order assemblages are also completely specified by the designer. Such 
robots cannot leam. 
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haviors in response to sensory input, or they learn a control function that modulates the 
sensory signals before they reach the basic behaviors. While these robots might work 
well, they are still subject to the errors and oversights of their designers in programming 
SMC into the functional base-level of the robot. 

How, then, does one enable a robot to learn SMC at the level of basic behaviors? 
There are at least two possibilities. 

1 . Design and implement on the robot the fundamental motor circuits that enable 
actuation. Have the robot move randomly while sensing. Reinforce any sen- 
sory-motor coupling (a temporal coincidence of sensory signals and motor ac- 
tions) that leads to purposeful motion, approach such as this is necessary for 
a fully autonomous agent, like an animal. This approach has been used suc- 
cessfully by researchers in artificial life [16]. Learning SMC this way with a 
robot could require much time. 

2. Take advantage of the fact that a robot can be teleoperated. When a person 
teleoperates a robot, the person's SMC causes the robot to act purposefully. If 
the robot records all of its sensory signals during repeated teleoperations, 
through signal analysis it should be able to identify the sensory-motor cou- 
plings that accompany purposeful motion. 

Both of these approaches require signal analysis algorithms that will detect signal 
correlations or coincidences. Moreover, the detected sensory-motor couplings must be 
used to construct basic behavior modules. Both of these problems are open research is- 
sues. 


ROBONAUT 

Robonaut is NASA's most sophisticated humanoid system (See figure 1). Its me- 
chanical systems have been designed to operate within the conditions of space in low- 
earth orbit. The Robonaut upper body is an articulated torso with two dexterous arms, 
left and right five-fingered hands, and a head with cameras mounted on an articulated 
neck, packaged in less volume than an Astronaut’s EMU. Robonaut is fully functional at 
the level of motor control and is operated via full-immersion VR. It has a large set of 
proprioceptic sensors and an active pan-tilt stereo vision system. Robonaut brings to the 
project, a high degree of dexterity, sophisticated teleoperability, and a rich sensor suite, 
but no autonomy. At this point in time it only works through teleoperation 
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Figure 1. Left: Robonaut; right: Robonaut’s hand. 


EXPERIMENTS 


The objective of the research began during the summer of 2000 is to enable Ro- 
bonaut to learn the sensory-motor control couplings that define a canonical set of basic 
behaviors and to learn the sensory signals that precede and follow behavior changes dur- 
ing task execution. The approach is to have a person teleoperate the robot through a task 
a number of times while the robot records the motor control sequence and the signals 
from its sensors. For the experiments reported herein, Robonaut' s task was to find, to 
reach toward, and to grasp one stationary object followed by another across the work- 
space. This was accomplished through teleoperation, wherein the teleoperator controlled 
the action of the robot through a full-immersion VR station. The signals recorded were 
the end-effector position and the 6-axis force and torque above the wrist on the forearm. 



Figure 2. Left: end-effector position; middle: force on wrist; right: torque on wrist 


LEARNING SMC 

The motor control sequence within each trial will be used to determine the motor 
events — the times of transition between continuous motor operation states. The motor 
events from a trial will be used to partition all the sensory signals within that trial. Since 
the same task is repeated by the same operator several times there should be the same 
number of motor events in each trial, although the time between them will vary. After all 
the trials are completed, the signals will be time warped to align the motor events across 
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trials. Then in a time interval bracketing the motor event, the signals from a single sensor 
will be correlated across all trials to determine if there is a corresponding sensory event 
(the signal exhibits a change consistently near the motor event.) Only the signals that ex- 
hibit a consistent sensoiy event within an interval of a motor event will be considered to 
be salient to that motor event and analyzed further. (A signal that is constant or that 
changes inconsistently near a motor event across multiple trials of the same task is pre- 
sumed to be superfluous to the SMC of that event.) Through averaging (or some nonlin- 
ear combination such as median filtering) a characteristic signal for that sensory event at 
the given motor event will be formed. Then the signals from different sensors will be 
correlated within individual trials to determine which sensors react together near the mo- 
tor events. To each motor event, the characteristic signals from the salient sensors are 
coupled to form a sensory-motor coordination event. An SMC event is, therefore, a mo- 
tor state transition, that is either preceded or followed by a consistent signals in more than 
one sensor. 


CONCLUSIONS AND FUTURE WORK 

At the time of this writing, the first experiments in SMC data gathering during 
teleoperation had been performed. We found that the teleoperation procedure is repeat- 
able in the way needed for the analysis: Having the same number of motor events yet 
having sufficient variability to detect true sensory events and to average out the spurious 
ones. It remains to perform the analysis described herein. 
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